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The  research  conducted  under  Grant  Number  AFOSR-83-0210  during  the  period 
15  July  1983  to  14  June  1988  is  partially  documented  in  [l],  [6],  [10],  [11],  [12],  [13], 
[14],  [15],  and  [16]. 

This  research  is  concerned  with  the  computational  and  theoretical  study  of  so¬ 
lution  techniques  for  significant  optimization  problems  involving  functions  that  are 
not  everywhere  differentiable.  In  addition  to  being  nonsmooth  the  functions  defining 
such  problems  are  often  only  implicitly  defined.  For  example,  the  evaluation  of  the 
objective  function  may  involve  solving  a  subproblem  that  is  itself  an  optimization 
problem.  An  example  of  this  type  of  problem  is  the  problem  of  minimizing  the  maxi¬ 
mum  eigenvalue  of  a  symmetric  matrix  whose  elements  depend  on  decision  variables. 
Other  nonsmooth  optimization  (NSO)  problems  arise  from  applying  decomposition, 
relaxation,  duality,  and/or  exact  penalty  function  techniques  to  large  or  complicated 
mathematical  programming  problems.  These  problems  involve  nonsmooth  functions 
such  as  absolute  value  functions,  maximum  value  functions  and,  more  generally,  sums 
of  maximum  value  functions. 

Having  an  efficient  and  reliable  method  for  solving  implicitly  defined  nonsmooth 
problems  gives  a  user  flexibility  in  modeling  a  problem  and  in  writing  function  eval¬ 
uation  subroutines  and  the  ability  to  exploit  parallel  processing  in  computation. 

For  the  case  of  constrained  minimization  of  a  function  of  a  single  variable  where 
generalized  derivatives  are  known  this  research  has  produced  a  rapidly  convergent 
algorithm  [10]  and  a  corresponding  reliable  and  easy-to-use  FORTRAN  subroutine, 
called  PQ1  [13,  17].  The  problem  functions  may  be  nonsmooth  and/or  nonconvex. 
The  method  converges  to  stationary  points  of  problems  for  which  secant  and/or  cut¬ 
ting  plane  methods  may  fail  and  it  converges  rapidly  on  some  problems  for  which 
these  other  methods  do  not  exhibit  superlinear  convergence.  The  new  safeguarding 
development  that  insures  convergence  via  a  sequence  of  bracket  intervals  with  de¬ 
creasing  lengths  does  not  detract  from  better  that  linear  convergence  and  fits  in  well 
with  numerical  practice,  because  it  keeps  apart  points  whose  difference  appears  in  the 
denominator  of  a  divided  difference  formula  for  second  derivative  approximation.  Our 
new  automatic  penalty  technique  for  handling  a  constraint  is  also  numerically  sound, 
because  it  is  independent  of  constraint  scaling  and  numerically  well-conditioned  un¬ 
der  a  mild  constraint  qualification.  This  algorithm  maintains  two  points  that  define 
a  bracket  containing  the  problem  solution.  Via  quadratic  and  polyhedral  approxima¬ 
tion  each  iteration  determines  a  new  point  that  becomes  a  new  bracket  endpoint  and, 
hence,  defines  a  new  bracket.  A  recent  result  submitted  for  publication  in  [16]  states 
that  either  the  new  point  is  superlinearly  closer  to  the  solution  that  both  current  _ 
bracket  endpoints  or  the  length  of  the  new  bracket  is  superlinearly  shorter  than  that  f 
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of  the  current  bracket.  This  is  proved  under  very  weak  convergence  rate  assumptions. 
This  work  was  presented  in  October  1987  at  the  SIAM  35th  Anniversary  Meeting  in 
Denver. 

In  order  to  show  the  efficacy  of  this  method  and  the  corresponding  FORTRAN 
code  PQl  was  used  to  solve  both  smooth  [13]  and  nonsmooth  [12]  versions  of  a  prac¬ 
tical  single  resource  allocation  problem  [5]  having  five  bounded  decision  variables  via 
a  dual  (min-max)  technique.  This  application  employed  PQl  in  a  nested  manner,  i.e. 
a  single  variable  “outer”  problem  was  solved  where  each  function  evaluation  involved 
solving  a  five  variable  “inner”  Lagrangian  problem.  The  inner  problem  separated 
into  five  independent  single  variable  problems  that  could  have  been  solved  in  par¬ 
allel.  The  original  five  variable  primal  problem  is  smooth.  To  make  a  nonsmooth 
version  the  five  single  variable  functions  in  the  objective  were  approximated  with 
piecewise  affine  functions.  Both  versions  were  solved  with  about  the  same  amount 
of  computational  effort  which  was  about  half  the  effort  required  in  the  best  run  of 
some  popular  nonlinear  programming  codes  applied  to  the  original  primal  problem 
[5].  These  results  were  reported  at  the  SIAM  Conference  on  Numerical  Optimization 
held  at  Boulder,  CO  in  June  1984,  the  NATO  Advanced  Study  Institute  on  Com¬ 
putational  Mathematical  Programming  held  at  Bad  Windsheim  in  July  1984  and  at 
the  IIASA  Workshop  on  Nondifferentiable  Optimization  held  at  Sopron  in  September 
1984. 

A  Masters  degree  student  at  WSU,  Han  Lim,  used  PQl  in  a  nested  manner  to 
obtain  maximum  likelihood  estimates  of  the  three  parameters  in  the  Weibull  proba¬ 
bility  density  function  [18]. 

In  a  private  communication  K.G.  Murty  has  informed  the  principal  investigator 
that  the  above  algorithm  will  be  described  in  a  new  book  he  is  writing.  Also,  as  did 
a  referee  of  [13],  Murty  has  reported  very  successful  use  of  PQl  by  colleagues  at  the 
University  of  Michigan. 

A  modification  of  this  subroutine  was  written  for  C,  Lemarechal  (INRIA,  France) 
for  performing  line  searches  required  by  n-variable  optimization  methods.  It  has 
worked  well  for  him  and  in  the  multivariable  research  described  below. 

A  BASIC  language  version  of  the  method  appears  in  [6].  It  was  tested  on  the 
problem  of  finding  a  root  (zero)  of  a  function  on  an  interval  by  minimizing  the  ab¬ 
solute  value  of  the  function  subject  to  a  constraint  defining  the  interval. 

Recent  joint  work  with  J.-J.  Strodiot  (Namur,  Belgium)  has  produced  theoreti- 
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cally  satisfying  and  practically  useful  ideas  for  solving  single  variable  minimization 
problems  using  function,  but  not  derivative,  values.  The  work  on  a  safeguarded 
bracketing  technique  and  on  quadratic  approximation  in  [15]  will  appear  in  Math¬ 
ematical  Programming ,  Vol.  43,  No.  2.  Also,  this  effort  has  produced,  what  is 
probably,  the  first  instance  of  a  function-value-only  method  for  nonsmooth  functions 
with  proven  rapid  convergence.  The  corresponding  paper  [14]  is  under  revision  in 
order  to  append  some  figures  to  illustrate  various  cases  mentioned  in  the  introduction 
and  considered  in  the  proofs  that  are,  unfortunately,  rather  lengthy  and  complicated. 

In  [15]  a  very  general  safeguarded  bracketing  technique  is  introduced  which  guar¬ 
antees  that  the  algorithm  iterates  are  sufficiently  distinct  and  that  the  iterate  se¬ 
quence  converges  to  a  stationary  point  for  locally  Lipschitz  functions.  When  this 
technique  is  combined  with  certain  sequential  polynomial  and/or  polyhedral  fitting 
methods  it  preserves  certain  types  of  rapid  convergence  that  are  properties  of  the 
fitting  methods  if  they  happen  to  generate  distinct  iterates.  Each  bracket  has  an 
interior  point  whose  function  value  does  not  exceed  that  of  those  of  the  two  bracket 
endpoints.  The  safeguarding  technique  consists  of  replacing  the  fitting  algorithm’s 
iterate  candidate  by  a  close  point  whose  distance  from  the  three  bracket  points  ex¬ 
ceeds  a  positive  multiple  of  the  square  of  the  bracket  length.  Also  in  [15]  it  is  shown 
that  a  given  safeguarded  quadratic  fitting  algorithm  converges  in  a  certain  better 
than  linear  manner  with  respect  to  the  bracket  endpoints  for  a  strongly  convex  twice 
continuously  differentiable  function. 

In  [14]  an  algorithm  for  nonsmooth  problems  is  introduced  which  uses  function 
values  at  five  points  to  generate  each  iterate.  It  employs  polyhedral  approximations 
as  well  as  quadratic  ones  and  the  above  safeguard.  For  certain  piecewise  twice  con¬ 
tinuously  differentiable  functions  it  is  shown  that  the  method  has  the  type  of  better 
than  linear  convergence  defined  earlier  in  work  on  the  n-variable  problem  [ll]  dis¬ 
cussed  below. 

Preliminary  versions  of  the  results  in  [14]  and  [15]  were  presented  at  the  Cam¬ 
bridge  Optimization  Symposium  held  in  England  in  March  1985  and  at  the  TIMS/ORSA 
Joint  National  Meeting  at  Los  Angeles  in  April  1986. 

The  groundwork  for  developing  an  efficient  method  for  solving  n-variable  im¬ 
plicitly  defined  nonsmooth  unconstrained  minimization  problems  is  established  in 
[11].  In  this  paper  a  new  concept  of  rapid  convergence  called  ubetter  that  linear 
convergence”  is  defined.  Also,  the  framework  of  a  second  order  method  based  on 
piecewise  quadratic  approximation  for  obtaining  this  type  of  convergence  is  given. 
The  method  stores  and  updates  several  Hessian  matrices  that  are  used  to  generate 
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data  for  quadratic  programming  search  direction  finding  subproblems.  It  also  in¬ 
cludes  a  safeguarding  technique  to  insure  first  order  convergence  on  problems  with 
semismooth  [7]  functions.  The  advantage  of  the  form  of  the  safeguard  is  that  it  does 
not  prevent  better  that  linear  convergence  from  occurring  on  problems  with  an  un¬ 
derlying  piecewise  C2  structure.  A  FORTRAN  code  to  test  these  ideas  and  guide 
future  research  has  been  written  by  a  former  research  assistant,  David  Elwood.  Some 
preliminary  results  of  this  research  were  presented  at  the  May  1987  SIAM  Conference 
on  Optimization  in  Houston. 

The  above  mentioned  work  for  the  unconstrained  case  was  extended  to  the  con¬ 
strained  case  in  the  thesis  [4]  of  the  principal  investigator’s  Ph.D.  student  Nilama 
Gupta.  In  [4]  better  than  linear  convergence  is  obtained  for  constrained  problems 
having  a  strongly  convex  constraint  function  and  a  corresponding  constraint  set  with 
a  strict  interior.  This  is  accomplished  via  an  n-variable  generalization  of  the  scale- 
free  automatic  penalty  technique  introduced  in  [9]. 

Elwood’s  FORTRAN  code  mentioned  above  uses  the  general  purpose  quadratic 
programming  code  QPSOL  [3]  to  solve  the  search  direction  finding  subproblems.  For 
efficiency  it  might  be  beneficial  to  employ  a  code  based  on  a  specialized  algorithm  in 
the  thesis  [1]  of  the  principal  investigator’s  former  Ph.D  student  Amal  Al-Saket.  Her 
work  is  an  extension  of  the  numerically  stable  constrained  least  squares  algorithm 
in  [8]  to  problems  with  more  general  quadratic  objectives.  It  also  contains  a  special 
procedure  for  resolving  degeneracy.  Two  papers  based  upon  the  results  in  [l]  are 
being  prepared  for  publication  submission. 

Incidentally,  the  concept  of  a  semismooth  function  mentioned  above  and  devel¬ 
oped  under  a  previous  grant  (AFOSR-74-2695)  continues  to  be  important.  For  ex¬ 
ample,  R.W.  Chaney  uses  it  in  developing  second  order  optimality  conditions  for 
nonsmooth  optimization  (See  [2]  and  references  therein). 
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